Limited Memory Bfgs Updating in a Trust–region Framework

The limited memory BFGS method pioneered by Jorge Nocedal is usually implemented as a line search method where the search direction is computed from a BFGS approximation to the inverse of the Hessian. The advantage of inverse updating is that the search directions are obtained by a matrix–vector multiplication. In this paper it is observed that limited memory updates to the Hessian approximations can also be applied in the context of a trust–region algorithm with only a modest increase in the linear algebra costs. At each iteration, a limited memory BFGS Step is computed. If it is rejected, then we compute the solutions for trust-region subproblem with the trust-region radius smaller than the length of L-BFGS Step. Numerical results on a few of the MINPACK-2 test problems show that the initial limited memory BFGS Step is accepted in most cases. In terms of the number of function and gradient evaluations, the trust-region approach is comparable to a standard linesearch implementation.